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A PROCEDURE FOR MANAGING INTERDISCIPLINARY 
INTELLIGENCE PRODUCTION 

I (b)(3)(c) I 


The most popular buzz-word in intelligence production circles these days is 
“interdisciplinary.” One hears a great deal about the needs of intelligence consumers, 
for integrated products — reports that pull together the economic, political, military, 
technological and societal aspects of an issue so that a policy maker gains a broad un- 
derstanding of it. This is necessary, it is argued, because policymakers themselves have 
no time to perform the integration. Hence, intelligence reporting that focuses on only 
one dimension of an issue at best can provide some useful, but incomplete, 
information; at worst it may be dismissed by the consumer as irrelevant. 

That one of the functions of intelligence is to provide policymakers with complete 
and relevant information on an important issue is obvious; the problem has been how 
to organize and manage the production of such information. This article describes one 
method that has been used with some success in the National Foreign Assessment 
Center (NFAC) and has now been institutionalized in the research planning process. 
My purpose is to record the lessons that we learned in producing a major interdisci- 
plinary study on the development of the prospects for Soviet military power so that 
those charged with managing issue-oriented research can profit from our experience. 
(Those more interested in our substantive findings can refer to the study. The 
Development of Soviet Military Power: Trends since 1965 and Prospects for the 
1980s.) 

Origins of the Study 

For several years it has been evident to the Intelligence Community that the 
environment for Soviet decisionmaking on national security issues is changing 
markedly. 

— Growth of the Soviet economy is slowing and is likely to decline further in the 
1980s as energy and manpower shortages compound the longer-standing 
problem of poor productivity. 

— Many of the top military and political leaders will be replaced soon: most are 
in their 60s and 70s and a number are in poor health. 

— Growing Western concern about Soviet military power, China’s opening to 
the West, and instability in Eastern Europe and the Third World are altering 
the international setting. 

— Advances in technology are providing new opportunities to upgrade military 
capabilities. 
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These complicating factors make it particularly difficult for intelligence analysts 
to forecast Soviet behavior. But the tools available to analysts have become more 
powerful: 

— The Office of Strategic Research (OSR), drawing on analysis performed 
throughout the Intelligence Community, has compiled an all-source data base 
on trends since the 1950s in Soviet military production, order-of-battle, 
manpower, and expenditures. 

— The Office of Scientific and Weapons Research (OSWR) has gained valuable 
insights into the Soviet system for managing missile and space programs and 
the development capacity of the Soviet aerospace industry. 

— The Office of Economic Research (OER) has developed new models to assess 
and predict Soviet economic performance. 

— The Office of P olitic ^ ^ alysis (OPA) has instituted research projects aimed 

at ' '' ' soviet policymakers and their possible 

views on policy issues. , 

r 

— Analysts throughout the community— in OSR, OSWR, OPA, and in DIA and 
the military services— have greatly improved their knowledge of the Soviet 
policy process. 

It was this coincidence of an important policy issue and new analytical tools that 
led OSR to initiate the process that led to production of a major interdisciplinary 
assessment of the prospects for Soviet military power. 

The Process 

The process began in early 1979 with planning for a two-day seminar on “Soviet 
Defense Decisionmaking. At that seminar analysts from the NFAC production offices 
briefed each other and their managers on their current understanding of Soviet 
decisionmaking processes and on the factors likely to influence Soviet choices on 
military forces and policies for the 1980s. Out of the discussions a consensus emerged 
in favor of a broadly-based, interoffice research effort aimed at producing an 
integrated report on the prospects for the development of Soviet military power in the 
coming decade. The group also identified the major research themes that the 
individual offices should investigate. 

Following the seminar, an interoffice working group was formed to turn the 
general research themes into a concrete plan of action. (I chaired the group; other 
members were j (h)(2)(r.) of OSWR; Q 

of QPA ; of OERrand 

pf OSR.) The working group was tasked with evaluating the current NFAC re- 
search program, recommending redefinition and rescheduling of planned research, 
and proposing new projects to meet the goals of the plan. 

Initially, some members of the working group expressed reservations about the 
plan, and some managers resisted the notion that an interoffice group should play a 
role in determining their production priorities. We also touched some sensitive nerves 
by recommending a few projects that the offices, for one reason or another, were less 
anxious to undertake. Eventually, however, we hammered out a consensus at the 
working group level. 

The working group codified its research proposals in a formal document, signed 
by all of the NFAC office directors, and set up a management information system to 
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keep the directors aware of the program’s progress. The program Dlan consisted of an 
interim report on recent trends in Soviet military forces, som(b)(1 ),uilding block 
papers on individual factors affecting Soviet policy, and an overview report for 
publication in the spring of 1981. 

By the summer of 1980, enough work on the building blocks project had been 
completed to warrant a working-level seminar to outline the major theses of the 
overview paper, and identify areas for further research. During October and 
November, a first draft of the overview paper was put together. Coordination of the 
paper was completed in March and in May the DCI transmitted the report to the 
President, Vice President, Secretary of State, Secretary of Defense and Assistant to the 
President for National Security Affairs. Several hunderd other consumers received the 
paper through normal dissemination channels. 

Reaction to the paper has been highly favorable. Several high-level recipients 
have praised the integrative character of the analysis. The report has contributed to 
debates on US military programs and has been made required reading for seminars at 
one of the war colleges. 

r' -■■■ 

Lessons Learned ' } 

In the process of producing our paper we learned a great deal about planning and 
managing interdisciplinary research. First, we learned that it is possible, within a 
hierarchical organization, to carry out an integrated research program. But we also 
learned that managing such a program is difficult and demanding: both careful 
planning and vigorous follow-up are vital. 

More specifically, we found that there are a number of essentia] steps in an 
interdisciplinary project: 

— Choosing the issue properly. 

— Sharing information and views among analysts with different backgrounds. 

— Defining the issue in terms of its major research themes. 

Appointing a project leader and working group to develop a program plan. 

— Assessing the adequacy of currently planned research in answering key 
questions about the research themes. 

Preparing a program blan with milestones for completion. 

— Formal commitment of resources to the effort. 

Monitoring the progress of the research program and taking action to keep it 
on track. 

Integrating the findings into an overview paper. 

As a result of our experiences, I believe that all of these steps are needed, 
regardless of the organizational structure established to carry out interdisciplinary 
analysis. 

Issue Identification 

The first order of business is to frame the issue so that it can be an effective guide 
to research. If this isn t done properly, the project team may be unable to assess the 
effectiveness of its plan. In the words of psychologist David Campbell, “If you don’t 
know where you re going, you re likely to end up somewhere else.” 

A particular challenge in this first step is to be sure that you are dealing with an 
issue. It’s easy to confuse the process of defining research to address an issue or answer 
a question with the (equally important) problem of ensuring that there is adequate 
analytical coverage of a topic or geographic area. An issue like “the ability and 
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willingness of the NATO allies to shoulder a larger defense burden” can be addressed 
by the method described in this article. But “Africa” and “Nuclear Proliferation” are 
areas and topics, not issues, and different techniques probably are required to manage 
research dealing with them. A good test is to try to frame the issue in terms of a ques- 
tion that can be addressed in a single paper. If you can’t, it probably isn’t an issue. 

Information Sharing 

This is an indispensable step in an interdisciplinary research effort. It is necessary 
to combat a common fallacy — that the information held by an individual or group is 
generally known by others. (This is rarely the case, even when individuals or 
organizations have close working contacts.) It is also important to have a common basis 
for planning the research program. 

Successful information sharing, like the other steps in an interdisciplinary 
research program, requires careful planning. 

— The atmosphere must.be right. We found that holding a two-day initial 
planning seminar in a remote location permitted an intensive exchange -Qf 
ideas away from day-to-day pressures. v 

— The briefing topics must be well-chosen. They should provide a Vide range of 
perspectives on the issue, but they must be relevant. There may be a tendency 
to brief on a topic not related to the issue simply because the information is 
available. This should be resisted. 

— The group dynamics must work. The conference planners must get the right 
mix of personalities and expertise, even if this results in an imbalance in 
organizational representation. They must also, however, ensure that the key 
analysts who will be working on the program and managers from the 
participating organizations are present. This is needed to ensure from the 
beginning a commitment to the goals of the program. 

Theme Definition 

The culmination of the initial sharing of information should be the identification 
of themes for organizing the research effort. If this is not done, there is a danger that 
participants in the information sharing phase will leave, having been entertained with 
briefings but without a clear idea of where the effort will lead. 

This is a tricky phase of the process, and the planners must be careful to explain 
its purpose to the participants. There will be a tendency for many to want to argue 
substance or draw conclusions, but this is not the time. Instead, the participants should 
strive to understand all of the facets of the issue and to record their understanding as a 
set of themes for further investigation. 

A technique that we used successfully was a variation of brainstorming called 
“funneling applied creativity.” After immersing seminar participants in substantive 
briefings for a day and a half, we solicited from them their ideas on research themes 
that should be followed up. The ground rules permitted anyone to make a proposal 
and prohibited any criticism or discussion until all of the suggestions had been put for- 
ward and recorded on flip charts. We then opened the floor for discussion and deleted 
some themes and refined others. The themes that we settled on were: 

— Soviet defense decisionmaking and management. 

— Soviet military goals, threat perceptions and requirements.. 

— Defense and the Soviet political succession. 

— The Soviet defense-industrial establishment. 

— Defense and the Soviet economy. 
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— Trends in Soviet military forces. 

— Soviet military technology. 

Appointing a Management Team 

The final act of the initial planning seminar should be to appoint a management 
team. The team will have two tasks: 

— To formulate a program plan. 

— To monitor and report on execution of the plan. 

The team should include representatives of all organizations that have substantive 
responsibilities related to the research issue. But more importantly, it must have the 
right blend of personalities and expertise. A critical task for managers, then, is to select 
a knowledgeable group of senior analysts or first line managers who know both the 
substantive issues and the research and production capabilities of their organizations. 

Assessment of Current Research 

Once there is agreement on the definition of the research issue and its themes, the>" 1 
first task for the management team is to assess the adequacy of current plg/is. The dif- 
ficult parts of this step are defining the criteria for measuring adequacy and finding 
out exactly what the current plans were. 

For our measuring stick we used a set of key questions structured into the seven 
research themes (see box below for examples). The concept was that research would be 
adequate if it would permit confident answers to all the key questions. To uncover in- 
formation about current plans, we found personal contacts to be far more useful than 
published research programs. (The published programs were often cryptic, out of date 
and unrealistic.) For this reason, it is important that the members of the working group 
be in a position to find out what really is going on in their offices and to provide realis- 
tic assessments of the intended scope and completion dates of currently scheduled 
projects. It is particularly important to go beyond the titles of planned research 
projects and find out more about their purposes and substantive theses. Without this 
information, it is impossible to judge the appropriateness of the planned research. 


Research Theme 3: Defense and the Soviet Political Succession 

Key Questions: What are the possible scenarios for the Soviet political 
succession. Who are the likely candidates for top leadership posts? What are 
their views and preferences in the area of defense resource allocation and 
mi itary policy? How much flexibility will a new regime have to alter resource 
priorities and policies? 


Research Theme 6: Trends in Soviet Military Forces 

Key Questions: What have been the key trends in Soviet military forces? 
Are they consistent across service and mission lines? What programs are now 
under way or known to be planned? When will they impact on force structure 
and capabilities and to what extent? 

What deficiencies do the Soviets perceive in their forces? What can we 
infer from this about new programs initiatives for which we currently have no 
evidence? 
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Preparation of a Plan 

Once the criteria for success are established and currently programmed research 
is identified, the project leader and management team can begin to prepare a program 
plan. The first step is fairly mechanical — array the scheduled projects by theme, 
compare their estimated completion dates, and identify necessary adjustments. Some .s 
projects might logically be combined or made into joint projects. 

For those key questions that will not be adequately answered by currently 
planned projects, the project manager and his group should prepare project proposals. 

These should be specific enough to be a basis for resource decisions by office 
managers. They should include, for example, a statement of the scope and proposed 
schedule, level of analytical effort and any special requirements such as ADP or 
contractor support. It’s useful to identify the analysts (or at least the branches) best 
suited to carry out each proposed project. 

At this stage, the management team should also identify requirements for 
overview papers to integrate the findings of the building-block projects. If possible, tKd" 
group should identify potential drafters so that they can be involved ip. the entire re- 5 
search program. 

After all of this homework is done, the project manager should prepare a 
consolidated program that identifies all of the building-block and overview papers, 
sets forth schedules and assigns responsibilities, and recommends new projects to fill 
gaps in current plans. 

Formal Commitment of Resources 

After the plan has been prepared in draft, it should be submitted for approval by 
senior management. The approval should be formal and specific. To make a program 
work, it is necessary that managers at all levels assign sufficient analytical resources 
and protect them. When approving the plan, managers should be made aware that 
they have made commitment to deliver their projects and that their own supervisors 
will be apprised of the progress of research. 

Once the plan is completed it should be circulated to all analysts and managers 
involved. (We gave our plan a little extra dignity by having it bound in hard covers.) It 
is essential that the participants understand their role in the plan. One of the 
important functions of a program plan is to alter the directions of individual projects 
so that they are responsive to the key questions. This can’t be done unless people know 
what the questions are and how their own work relates to them. The only way to en- 
sure this is good communications. It’s helpful to record in the Advance Work Plans of 
both analysts and managers that their projects are related to an interdisciplinary 
research issue. 

Monitoring and Troubleshooting 

Even the most carefully prepared plan will fail without aggressive follow-up. 
Periodic reports to senior managers are essential to keeping the program on track. (We 
sent reports quarterly.) The project manager and team must be frank in identifying 
problems and pressing for their solution. We found that most problems solved 
themselves once they were made known. But occasionally we had to keep reminding 
managers of their commitments and on one occasion had to convene a formal meeting 
of office directors. 
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Another essential ingredient is an effort to promote better communications 
among the analysts working on the building-block projects. Merely publishing a 
program plan will not make analysts aware of their responsibilities and their 
importance in the overall process. It will be necessary to organize briefings, seminars, 
and retreats to facilitate communications, and the members of the working group must 
be prepared to spend a great deal of time and effort trying to break down 
bureaucratic and disciplinary barriers. 

Production and Evaluation 

The final step in the process is to integrate the results of the building-block 
projects into a coherent overview paper. The danger in this stage is that it may 
degenerate into “analysis by staple.” Interdisciplinary analysis must be more than 
editing inputs from various specialists; it should entail a real effort to combine the 
methods of several disciplines to create new insights. 

It’s particularly important to select the right drafting team. It’s not enough to 
form a group of, say, an engineer, an economist, and a political scientist. At times peo- 
ple from different specialties have such different mindsets and vocabularies that they 
literally cannot communicate. Ideally, each drafter should have experience in at least 
two disciplines. And whatever their background the drafters must work effectively 
together — have the right body chemistry. 

We found that the close personal relationships that were forged during the 
working group’s seminars were an important ingredient in promoting interdisciplinary 
analysis. The participants knew each other, appreciated the work of other organiza- 
tions, and learned the concepts and vocabulary of other analytical disciplines well 
enough to communicate effectively. Moreover, most coordination problems were 
cleared up during preparation of the building-block papers, leaving only a few 
contentious issues for the overview report. 

This is not to say that we had no problems. We faced two serious substantive dis- 
agreements. Opinion was sharply divided between OSR and OER on prospects for 
Soviet defense spending and between OSR and OPA on the extent to which Soviet 
force procurements were driven by military or political considerations. These issues 
were to have been addressed in building-block papers, but the schedules for the papers 
had slipped so that coordinated conclusions were not available for the overview report. 
Eventually we solved the problems — the first by more analysis and the presentation of 
alternative projections; the second by textual changes and a realization that the 
difference was largely semantic. But it would have been more efficient to identify the 
differences of opinion earlier, in the building-block projects, so that they could be 
worked out in a more rigorous fashion. 

The process of managing interdisciplinary research should not stop with publica- 
tion of an overview paper. It’s necessary, then, to have a formal evaluation of the pro- 
gram’s accomplishments and of the remaining gaps in knowledge. The mechanism we 
used was yet another two-day seminar at which we reported to NFAC managers on 
what they had received for their resource commitment and solicited their help in 
identifying and sponsoring follow-on research. 


Final Thoughts 


The process described in this article worked. There may be other ways of 
producing a major interdisciplinary report, but based on my experience I believe there 
are five indispensable ingredients. 
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— Commitment by senior managers to the importance of the issue and a 
willingness to devote resources to it and protect those resources. 

— Careful planning that involves all interested parties from the beginning. 

— Good communications both up the line to management and across organiza- 
tional lines among analysts. 

— The right body chemistry among planners and analysts. 

— Follow-up to ensure that all participants retain their commitment and carry 
out their responsibilities. 

There are commodities to be rationed carefully among issues that are genuinely 
high in priority. My own opinion is that an organization the size of NFAC probably 
should not undertake more than about half a dozen major interdisciplinary projects at 
a time. A larger number would dilute the commitment of senior managers and reduce 
their ability to protect the resources devoted to the programs. But if aggressively 
managed and strongly supported, the process I’ve described has a high probability pf 
payoff. r' 


“It is at all times necessary . . . that we frequently 
refresh our patriotism by reference to first principles. It is 
by tracing things to their origin that we learn to under- 
stand them, and it is by keeping that line and the origin 
always in view that we never forget them.” 


Tom Paine 
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